Skip to main content

​
Overview

Fetches real-time lists of stocks that have hit upper or lower circuit breakers (price freeze limits) using the Dhan ScanX Analytics API. Implements 3-tier fallback to Dhan’s Next.js API and web scraping. Source: fetch_circuit_stocks.py
Phase: Phase 2 (Enrichment)
Output: upper_circuit_stocks.json, lower_circuit_stocks.json

​
API Endpoint

POST https://ow-scanx-analytics.dhan.co/customscan/fetchdt

​
Request Payload

​
data.sort
string
default:"Mcap"
Sort field (Market Cap)
​
data.sorder
string
default:"desc"
Sort order (descending)
​
data.count
int
default:"500"
Maximum number of results to return
​
data.fields
array
required
List of fields to include in response
​
data.params
array
required
Filter parameters array
​
data.pgno
int
default:"1"
Page number

​
Example Request (Upper Circuit)

{
  "data": {
    "sort": "Mcap",
    "sorder": "desc",
    "count": 500,
    "fields": ["Sym", "DispSym", "Ltp", "PPerchange", "Mcap", "Volume", "High5yr", "Low1Yr", "High1Yr", "Pe", "Pb", "DivYeild"],
    "params": [
      {"field": "LiveData.UpperCircuitBreak", "op": "", "val": "1"},
      {"field": "OgInst", "op": "", "val": "ES"},
      {"field": "Seg", "op": "", "val": "E"}
    ],
    "pgno": 1
  }
}

​
Example Request (Lower Circuit)

{
  "data": {
    "sort": "Mcap",
    "sorder": "desc",
    "count": 500,
    "fields": ["Sym", "DispSym", "Ltp", "PPerchange", "Mcap", "Volume", "High5yr", "Low1Yr", "High1Yr", "Pe", "Pb", "DivYeild"],
    "params": [
      {"field": "LiveData.LowerCircuitBreak", "op": "", "val": "1"},
      {"field": "OgInst", "op": "", "val": "ES"},
      {"field": "Seg", "op": "", "val": "E"}
    ],
    "pgno": 1
  }
}

​
Configuration

scans_config = {
    "upper_circuit_stocks.json": {
        "val": "1",
        "field": "LiveData.UpperCircuitBreak",
        "web_key": "stocks/market/shares-with-upper-circuit",
        "web_url": "https://dhan.co/stocks/market/shares-with-upper-circuit/"
    },
    "lower_circuit_stocks.json": {
        "val": "1",
        "field": "LiveData.LowerCircuitBreak",
        "web_key": "stocks/market/lower-circuit-stocks",
        "web_url": "https://dhan.co/stocks/market/lower-circuit-stocks/"
    }
}

​
Function Signatures

​
get_build_id()

def get_build_id():
    """
    Dynamically fetch the Next.js buildId from Dhan website.
    
    Returns:
        str: Build ID string, or None if extraction fails
    """

​
fetch_circuit_stocks()

def fetch_circuit_stocks():
    """
    Main function that fetches both upper and lower circuit stocks using 3-tier fallback.
    Writes upper_circuit_stocks.json and lower_circuit_stocks.json to current directory.
    """

​
Output Structure

​
Symbol
string
Stock trading symbol
​
Name
string
Company display name
​
LTP
number
Last traded price
​
ChangePercent
number
Percentage price change
​
MarketCap
number
Market capitalization
​
Volume
number
Trading volume
​
High5Yr
number
5-year high price
​
High1Yr
number
1-year high price
​
Low1Yr
number
1-year low price
​
PE
number
Price-to-earnings ratio
​
PB
number
Price-to-book ratio
​
DivYield
number
Dividend yield percentage

​
Example Output

[
  {
    "Symbol": "TATASTEEL",
    "Name": "Tata Steel Limited",
    "LTP": 145.50,
    "ChangePercent": 19.95,
    "MarketCap": 178456.32,
    "Volume": 85234567,
    "High5Yr": 165.00,
    "High1Yr": 155.00,
    "Low1Yr": 95.00,
    "PE": 12.5,
    "PB": 1.8,
    "DivYield": 2.5
  }
]

​
Data Sources

The script implements a 3-tier fallback strategy:

​
Primary: ScanX Analytics API

Direct POST request to customscan/fetchdt endpoint (see payload above).

​
Secondary: Dhan Next.js JSON API

GET https://dhan.co/_next/data/{buildId}/{web_key}.json
Extracts stock list from nested pageProps structure.

​
Tertiary: Web Scraping

Fallback scraping from web URLs, extracting __NEXT_DATA__ script block.

​
Dependencies

  • requests — HTTP client
  • json — JSON parsing
  • re — Regex for buildId extraction
  • BeautifulSoup — HTML parsing for fallback scraping

​
Error Handling

  • 3-tier fallback ensures high reliability
  • 10-second timeout for all HTTP requests
  • Graceful degradation through each fallback layer
  • Prints detailed progress messages for debugging
if success:
    print(f"  Successfully found {len(cleaned_list)} stocks via ScanX API.")
else:
    print(f"  Critical failure: Could not fetch {filename}.")

​
Field Mapping

API response fields are mapped to consistent output format: 
cleaned_list.append({
    "Symbol": item.get('Sym'),
    "Name": item.get('DispSym'),
    "LTP": item.get('Ltp'),
    "ChangePercent": item.get('PPerchange'),
    "MarketCap": item.get('Mcap'),
    "Volume": item.get('Volume'),
    "High5Yr": item.get('High5yr'),
    "High1Yr": item.get('High1Yr'),
    "Low1Yr": item.get('Low1Yr'),
    "PE": item.get('Pe'),
    "PB": item.get('Pb'),
    "DivYield": item.get('DivYeild')
})

​
Usage Example

python3 fetch_circuit_stocks.py
Expected Output: 
Processing upper_circuit_stocks.json...
  Primary Fetch: ScanX Analytics API...
  Successfully found 23 stocks via ScanX API.
Processing lower_circuit_stocks.json...
  Primary Fetch: ScanX Analytics API...
  Successfully found 15 stocks via ScanX API.

​
Integration

This script is part of Phase 2 (Enrichment) in the EDL Pipeline. The output files are consumed by:
  • bulk_market_analyzer.py — Used for market sentiment analysis
  • Analytics dashboards for intraday monitoring
Run via master pipeline: 
python3 run_full_pipeline.py